While software systems continue to grow in size and complexity, business demands continue to require shorter development cycles. This has led software developers to compromise on functionality, time to market, and quality of software products. Furthermore, the increased schedule pressures and limited availability of resources and skilled labor can lead to problems such as incomplete design of software products, inefficient testing, poor quality, high development and maintenance costs, and the like. This may lead to poor customer satisfaction and a loss of market share for software developers.
To improve product quality, many organizations devote an increasing share of their resources to testing and identifying problem areas related to software and the process of software development. Accordingly, it is not unusual to include a quality assurance team in software development projects to identify defects in the software product during, and after development of a software product. By identifying and resolving defects before marketing the product to customers, software developers can assure customers of the reliability of their products, and reduce the occurrence of post-sale software fixes such as patches and upgrades which may frustrate their customers.
Software testing may involve verifying the correctness, completeness, security, quality, etc. of a product. During testing, a technical investigation may be performed by, for example, executing a program or application with the intent to find errors. If errors are found, one or more areas in the software code may be identified based on the errors. Therefore, developers may alter the code in the identified regions to obviate the error.
After a defect has been fixed, data regarding the defect, and the resolution of the defect, may be stored in a database. The defects may be classified and analyzed as a whole using, for example, Orthogonal Defect Classification (ODC). ODC is a commonly used complex quality assessment schema for understanding code related defects uncovered during testing.
It is widely accepted in the testing industry that the least expensive defects to fix are those found earliest in the life cycle. However, a problem in complex system integration testing is that there may be very few comprehensive opportunities for projects to remove defects cost effectively prior to late phase testing—and by that point in the life cycle (i.e., late phase testing), defects are relatively expensive to fix. Furthermore, for many projects, there are particular kinds of high impact exposures—defects in the area of security, for example—that are critical to find and fix, but are also difficult to test for.
There are numerous automated code inspection tools available on the market today designed to address this problem; however, for many projects, it is not cost effective for an organization to purchase licenses for all of the tools needed to cover all of the exposures of interest to them. Moreover, even if it was cost effective for an organization to purchase licenses for all of the tools needed to cover all of the exposures, there is no way to understand the return on this investment in terms of the impact on reducing the numbers of defects found in late phase testing and in production.
That is, no defect analysis schema capable of accurately measuring value received from performing specific automated code inspection activities is known to exist. For example, software defect analysis models are incapable of providing return on investment metrics, e.g., because they do not provide actionable recommendations. Thus, there is no way to understand the return on this investment (e.g., the purchase of code inspection services) in terms of the impact on reducing the numbers of defects found in late phase testing and in production. That is, the code inspection services reports (for example, from the plurality of code inspection services, e.g., specifically tailored for their project) do not interpret defects uncovered via the automated code inspection subscription service and do not provide actionable recommendations. Rather, such code inspect service reports, for example, only identify defects uncovered via the automated code inspection subscription service. As a result, an organization cannot measure the cost versus the benefit of taking an improvement action because the code inspection services report does not point to an improvement action in the first place. The code inspection services report, for example, cannot tell test managers whether or not their test effort is effective (whether they are finding the kinds of defects their test effort is designed to find) because the analysis models do not collect the information necessary to provide such insights. Thus, this automated code inspection subscription service does not allow projects to accurately assess the impact of automated code inspections on, for example, critical exposure areas and does not allow for effective planning of, for example, late phase testing and production support needs.
As a result, an organization, for example, cannot determine that a particular distribution of defects (e.g., as determined by the automated code inspection subscription service) indicates that the organization may need to devote more focus on shifting defect removal to earlier phases in the software development life cycle. Additionally, for example, an organization cannot determine an expected resulting defect distribution should the shifting of the defect removal to earlier phases be achieved. Because current defect analysis models fall short of demonstrating their value relative to their costs, organizations that could benefit the most from in depth code inspection analysis may not frequently leverage in depth code inspection analysis (e.g., for one or more projects).
Accordingly, there exists a need in the art to overcome the deficiencies and limitations described hereinabove.